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Abstract 

This  paper  deals  with  vision-based  closed-loop  control  schemes  for  collision  avoidance 
as  well  as  maintenance  of  clearance  in  a-priori  unknown  textured  environments.  These 
control  schemes  are  based  on  fuzzy  logic  and  employ  a visual  motion  cue,  we  call  the 
Visual  Threat  Cue  (VTC)  that  provides  some  measure  for  a relative  change  in  range  as 
weU  as  clearance  between  3D  surface  and  a fixated  observer  in  motion.  It  is  a collective 
measure  obtained  directly  from  the  raw  data  of  gray  level  images,  is  independent  of  the  3D 
surface  texture  and  needs  no  optical  flow  information,  3D  reconstruction,  segmentation, 
feature  tracking  or  preprocessing.  This  motion  cue  is  scale-independent,  rotation 
independent  and  is  measured  in  [time"^]  units. 

Design  of  a closed-loop  conventional  controller  for  vision  based  navigation  tasks  pose  a 
problem  as  the  system  is  complex  and  ill-defined.  On  the  other  hand  fuzzy  control  which  is 
closer  in  spirit  to  human  thinking  and  can  implement  linguistically  expressed  heuristic 
control  policies  directly  without  any  knowledge  about  the  dynamics  of  the  complex 
process.  The  fuzzy  controllers  were  tested  in  real  time  using  a 486-based  Personal 
Computer  and  a camera  capable  of  undergoing  6-DOF  motion.  Results  are  highly 
encouraging. 

Key  Words:  Active  Vision,  Collision  Avoidance,  Maintenance  of  Clearance,  Visual  Motion 
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1 Introduction 

When  dealing  with  a moving  camera-based  autonomous  navigation  system,  a huge 
amount  of  visual  data  is  captured.  For  vision-based  navigation  tasks  (such  as  obstacle 
avoidance,  maintaining  safe  clearance,  etc.),  relevant  visual  information  needs  to  be 
extracted  from  the  visual  data  and  used  in  real-time  closed-loop  perception-action  control 
system.  In  order  to  accomplish  safe  visual  navigation  several  questions  need  to  be 
answered,  including: 

1.  What  is  the  relevant  visual  information  to  be  extracted  from  a sequence  of  images? 

2.  How  does  one  extract  this  information  from  a sequence  of  2D  images? 

3.  How  to  generate  control  commands  to  the  vehicle  based  on  the  visual  information 
extracted? 

This  paper  provides  answers  to  all  three  questions  with  emphasis  on  the  third  one  i.e., 
generation  of  control  signals  for  collision  avoidance  and  maintenance  of  clearance  using 
visual  information  only. 

Usually  the  process  of  driving  or  flying  in  a 3D  environment  involves  a human 
operator.  The  operator  acts  in  part  as  a sensory  feedback  in  the  perception-action  closed- 
loop  control  to  ensure  safe  navigation  in  real  time.  It  becomes  a difficult  problem  to 
replace  the  human  operator  by  a vision-based  system  to  achieve  similar  tasks  for  the 
following  reasons:  In  outdoor  navigation  the  environment  is  usually  unknown  and 
unstructured,  and  the  same  3D  scene  may  result  in  many  different  images  due  to  changes 
in  illumination  conditions,  relative  distances,  orientation  of  the  camera,  choice  of  fixation 
point,  etc.,  as  well  as  various  camera  parameters  such  as  zoom,  resolution,  focus,  etc. 
There  is  a need  for  an  approach,  to  obtain  relevant  visual  information  about  relative 
proximity  in  the  presence  of  the  above  mentioned  factors  and  employ  it  as  a set  of  sensory 
feedback  signals  to  accomplish  the  tasks  of  safe  navigation. 

The  problem  of  automating  vision-based  navigation  is  a challenging  one  and  has  drawn 
the  attention  of  several  researchers  over  the  past  few  years  (see  for  example  [1-14]). 
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Usually  identifying  the  surrounding  object  is  not  important  for  such  tasks,  i.e.,  is  it  a tree, 
mountain  or  another  vehicle;  what  is  more  important  is  whether  a particular  object  is  an 
obstacle  or  not,  i.e.,  is  the  camera  on  a collision  course  with  it,  is  there  enough  clearance, 
etc.?  For  navigation  tasks  recovering  3D  scene  and  its  attributes  may  not  be  necessary  as 
it  may  contain  information  which  is  not  relevant  for  the  task  at  hand.  Visual  cues  such  as 
time-to-contact  [12]  and  looming  [10,  20,  21]  carry  important  information  about  the 
relative  proximity.  These  cues  do  not  need  any  reconstruction  process  which  is  usually 
computationally  intense  and  in  many  cases  are  sufficient  for  safe  navigation. 

It  is  well  established  in  the  literature  (computer  vision  as  well  as  psychology)  that 
optical  flow  plays  an  important  role  in  the  control  of  human  and  machine  behavior  in  the 
environment  [15-19].  The  extraction  of  optical  flow  from  a sequence  of  images  is  sensitive 
to  noise  in  images  and  needs  pre-processing  like  spatio-temporal  smoothing  [22-24]  which 
may  be  computationally  expensive.  Alternatives  to  optical  flow  information  as  sensory 
feedback  for  obstacle  avoidance  include  geometrical  properties  like  size,  shape,  contour 
and  area  of  image  entities,  imaged  texture  [25,  26],  focus  [28-30],  etc. 

A differential  invariant  of  the  image  field  based  visual  information  about  time-to- 
coUision  is  presented  in  [14].  In  [26]  variations  in  image  statistics  are  employed  to  extract 
the  four  components  of  an  affine  transformation.  A qualitative  view  of  the  use  of  these 
components  as  sensory  feedback  information  for  collision  avoidance  is  also  presented 
[26].  In  [27]  it  is  shown  that  the  relative  changes  in  edgels  of  visible  texture  in  a unit  area 
to  be  equal  to  looming  described  in  [10].  This  approach  of  using  edge  density  in  an  image 
is  an  alternative  to  the  use  of  flow  based  approach  to  extract  looming  which  is  very 
sensitive  to  noise. 

Fuzzy  control  is  closer  in  spirit  to  human  thinking  and  can  implement  linguistically 
expressed  heuristic  control  policies  directly  without  any  knowledge  about  the  dynamics  of 
the  complex  process.  Research  in  the  area  of  fiizzy  control  was  initiated  by  Mamdani's 
pioneering  work  [47],  which  had  been  motivated  by  Zadeh's  seminal  papers  on  fuzzy 
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algorithms  [48]  and  linguistic  analysis  [49.],  In  the  past  few  years  several  researchers  have 
addressed  the  use  of  fiizzy  control  for  ill-defined  processes  for  which  it  is  difficult  to 
model  the  dynamics  (see  for  example  [46,  50, 51]). 

This  paper  deals  with  vision-based  closed-loop  control  schemes  for  collision 
avoidance  as  weU  as  maintenance  of  clearance  in  a-priori  unknown  textured 
environments.  These  control  schemes  are  based  on  fiizzy  logic  and  employ  a visual  motion 
cue,  we  call  the  Visual  Threat  Cue  (VTC)  that  provides  some  measure  for  a relative 
change  in  range  as  weU  as  clearance  between  3D  surface  and  a fixated  observer  in 
motion.  It  is  a collective  measure  obtained  directly  from  the  raw  data  of  gray  level 
images,  is  independent  of  the  3D  surface  texture  and  needs  no  optical  flow  information, 
3D  reconstruction,  segmentation,  feature  tracking  or  preprocessing.  This  motion  cue  is 
scale-independent,  rotation  independent  and  is  measured  in  [time"^]  units. 

Design  of  a closed-loop  conventional  controller  for  vision  based  navigation  tasks  pose 
a problem  as  the  system  is  complex  and  ill-defined.  On  the  other  hand  fuzzy  control  which 
is  closer  in  spirit  to  human  thinking  and  can  implement  linguistically  expressed  heuristic 
control  policies  directly  without  any  knowledge  about  the  dynamics  of  the  complex 
process.  The  fuzzy  controllers  were  tested  in  real  time  using  a 486-based  Personal 
Computer  and  a camera  capable  of  undergoing  6-DOF  motion. 

This  paper  is  organized  as  follows:  In  section  2 an  overview  of  the  VTC  is  presented, 
details  of  the  control  schemes  are  presented  in  section  3,  in  section  4 we  present  the 
experimental  results,  section  5 presents  experimental  results  and  analysis  and  finally 
section  6 presents  conclusion  and  an  overview  of  future  work. 
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2 Overview  of  the  Visual  Threat  Cue  (VTC) 


We  review  a visual  motion  cue  called  the  Visual  Threat  Cue  (VTC)  [43]  that  provides 
some  measure  for  a relative  change  in  range  as  well  as  for  specific  clearance  between  a 
3D  surface  and  a fixated  observer  in  motion.  This  cue  is  independent  of  the  3D 
environment  and  needs  no  a-priori  knowledge  about  it.  It  is  time-based,  rotation 
independent  and  does  not  need  3D  reconstruction.  This  cue  can  be  extracted  directly  from 
the  raw  gray  level  data  of  images  and  does  not  need  optical  flow  information, 
segmentation,  feature  tracking  and  pre-processing.  Mathematically  the  VTC  (for  R > Rq) 
is  defined  as  follows: 


V7C  = 


d{R)ldt 

R{R-R,) 


where  R is  the  range  between  the  observer  and  a point  on  the  3D  surface,  d(R)/dt  is 
the  differentiation  of  R w.r.L  time  and  Rq  is  the  desired  minimum  clearance  and  has  the 
same  units  as  R.  Note  that  the  units  of  the  VTC  are  [time"^].  The  VTC  has  been  shown  to 
be  independent  of  the  rotational  motion  and  can  be  measured  without  knowledge  about 
R [43]. 

The  VTC  corresponds  to  a visual  field  surrounding  the  moving  observer,  i.e.,  there  are 
imaginary  3D  surfaces  attached  to  the  observer  that  are  moving  with  it,  each  of  which 
corresponds  to  a value  of  the  VTC.  The  points  that  lie  on  a relatively  smaller  surface 
corresponds  to  a relatively  larger  value  of  VTC,  indicating  a relatively  higher  threat  of 
collision.  The  VTC  value  on  the  minimum  clearance  sphere  of  Rq  centered  at  the  location 
of  the  observer  is  the  maximum  which  is  infinity,  indicating  that  the  absolute  distance 
between  the  observer  and  the  camera  is  the  minimum  clearance.  See  Figure  (1).  Note  that 
this  field  is  not  a sphere. 
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Based  on  this  knowledge  about  the  cue,  one  can  demarcate  the  region  around  an 
observer  into  safe,  high  risk  and  danger  zones  (see  Figure  (2)).  This  can  be  used  to 
directly  generate  control  action  tasks  such  as  collision  avoidance,  maintenance  of 
clearance,  etc.  The  VTC  is  related  to,  but  different  from,  the  time-to-contact  and  the 
looming  concepts  [10, 12,  20,  21]. 

A practical  method  to  extract  the  VTC  from  a sequence  of  images  of  a 3D  textured 
surface  obtained  by  2i  fixated,  fixed-focus  monocular  camera  in  motion  has  been  presented 
in  [43].  This  approach  is  independent  of  the  3D  surface  texture  and  needs  almost  no 
camera  calibration.  For  each  image  in  such  a 2D  image  sequence  of  a textured  surface,  a 
global  variable  (which  is  a measure  for  dissimilarity)  called  the  Image  (^ality  Measure 
(IQM)  is  obtained  directly  from  the  raw  data  of  the  gray  level  images.  The  VTC  is 
obtained  by  calculating  relative  changes  in  the  IQM.  This  approach  by  which  the  VTC  is 
extracted  can  be  seen  as  a sensory  fusion  of  focus,  texture  and  motion  at  the  raw  data 
level.  The  algorithm  to  extract  this  cue  works  better  on  natural  images  including  fractal- 
like images,  where  more  details  of  the  3D  scene  are  visible  in  the  images  as  the  range 
shrinks  and  also  can  be  implemented  in  parallel  hardware.  The  VTC  can  be  used  to 
directly  maintain  clearance  in  unstructured  environments. 
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2.1  Image  Quality  Measure  (IQM) 

Local  spatial  gray  tone  variations  in  an  image  give  rise  to  a visual  pattern  in  the  image 
known  as  texture.  These  spatial  gray  level  variations  are  due  to  the  visual  characteristics  of 
the  3D  scene  being  imaged,  the  illumination,  the  range  between  the  scene  and  the 
observer,  as  well  as  due  to  camera  parameters  like  zoom,  aperture,  resolution,  focus,  etc. 
When  there  is  a relative  motion  between  a textured  surface  and  a fixated,  fixed-focus 
moving  observer,  the  perceived  texture  in  the  2D  image  varies.  For  instance,  consider  the 
case  of  a camera  that  is  initially  focused  to  a 3D  surface  at  a very  short  distance  and 
gradually  moves  away  from  this  surface.  As  a result,  the  perceived  2D  image  texture 
varies  from  one  image  to  another,  mainly  due  to  focus,  i.e.,  the  image  of  the  scene  in 
perfect  focus  is  very  sharp  and  has  many  details,  then  as  the  camera  moves  away  from  the 
scene,  fine  details  gradually  get  smeared  and  eventually  disappear  (see  Figure  (3)).  When 
the  image  is  in  perfect  focus,  the  dissimilarity,  i.e.,  spatial  gray  level  variations  is  very  high, 
and  as  the  details  get  smeared  the  dissimilarity  gets  smaller  and  smaller.  We  describe  an 
IQM  to  measure  the  dissimilarity  of  the  image.  Using  the  relative  temporal  variations  in 
this  IQM  we  extract  the  VTC.  Next  we  present  a brief  overview  of  several  possible 
approaches  to  extract  the  dissimilarity  of  images,  and  describe  a practical  way  to  extract 
the  VTC  from  variations  in  the  IQM. 

The  area  of  texture  classification  has  drawn  the  attention  of  researchers  in  the  area  of 
computer  vision  over  the  past  two  decades  (see  for  example  [31-39]).  One  of  the  earliest 
areas  of  interest  in  the  texture  analysis  was  texture  segmentation  and  scene  analysis.  These 
approaches  may  be  broadly  classified  as  statistical  approaches  and  approaches  based  on 
structural  properties.  The  statistical  approaches  usually  employ  features  that  measure  the 
coarseness  and  the  directionality  of  textures  in  terms  of  the  averages  over  a windowed 
portion  of  the  image.  While  structural  methods  on  the  other  hand  describe  the  geometrical 
properties  like  size,  shape,  area,  etc.  of  the  objects  in  the  scene.  Structural  analysis  is 
suitable  for  structured  environments  and  also  needs  some  a-priori  knowledge  about  the 
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scene.  The  statistical  methods  that  are  in  use  are  mainly  classified  into  the  following 
categories: 

1.  Spatial  gray  level  dependence  methods  [31, 33, 34] 

2.  Spatial  frequency  based  methods  [33, 35] 

3.  Stochastic  model  based  features  [36, 37] 

4.  Heuristic  approaches  [38,  39] 

The  most  popular  being  the  spatial  gray-level  dependence  and  the  stochastic  model- 
based  approaches.  Usually  spatial  gray  level  based  approaches  are  probabilistic,  dependent 
on  the  number  of  gray  levels  in  the  image  and  computationally  expensive.  Stochastic 
model  approaches  are  dependent  on  the  model  of  the  texture.  Among  several  possible 
approaches  to  describe  the  quality  of  texture  in  an  image,  we  extended  an  approach 
presented  in  [40]  to  describe  the  dissimilarity  of  images  using  IQM.  The  advantages  of 
using  this  approach  over  the  other  approaches  are: 

1.  It  gives  a global  measure  of  quality  of  the  image,  i.e.,  one  number  which 
characterizes  the  image  dissimilarity  is  obtained. 

2.  It  does  not  need  any  preprocessing,  i.e.,  it  works  directly  on  the  raw  gray  level  data 
without  any  spatial  or  temporal  smoothing. 

3.  It  does  not  need  a model  of  the  texture  and  is  suitable  for  many  textures. 

4.  It  is  simple  and  can  be  implemented  in  real  time  on  parallel  hardware. 

5 It  is  non-probabilistic  and  is  independent  of  the  number  of  gray  levels  used  in  the 
image 


Mathematically,  the  IQM  is  defined  as  follows  [43]: 


1 yr 

x=x,y=y, 


( k L, 

X '^\Hxyy)-I(x  + p,y  + q)\ 

>=-L,  q=-L, 


\ 


s 


where  I(x,y)  is  the  intensity  at  pixel  (x,y)  and  xj  and  xf  are  the  initial  and  final  x- 
coordinates  of  the  window  respectively  ; yj  and  yp  are  the  initial  and  final  y-coordinates  of 

the  window  in  the  image  respectively  and  Lc  and  Lr  are  positive  integer  constants;  and  D 
is  a number  defined  as  D = ( 2L^  + l)x(2L^  + l)x( -Xi)x{  -y. ). 


2.2  Extraction  of  the  VTC  from  relative  variations  of  IQM 


Based  on  our  experimental  results  [43],  we  observed  that  relative  temporal  changes  in 


the  IQM  behave  in  a very  similar  fashion  to  the  VTC  , i.e.. 


diIQM)ldt  _ p d{R)ldt 


IQM 


= Ro 


R(R-Ro) 


The  VTC  is  independent  of  the  magnitude  of  the  IQM.  The  following  is  a sample  set  of 
five  images  (out  of  71)  that  corresponds  to  a texture  from  Brodatz  album  [42]  (see  Figure 
(3))  as  seen  by  a fixating,  moving,  fixed-focus  camera.  The  graphs  of  the  measured  IQM 
and  VTC  are  obtained  directly  from  the  images  without  additional  processing.  Note  that 
the  measured  IQM  (Figure  (4))  is  a "smooth"  function,  resulting  in  good  VTC  values. 
Very  similar  results  were  reported  in  [43]  for  twelve  different  textures  of  the  same  album 
[42]. 


Figure  3:  A sample  of  five  images  of  a textured  surface  (D1 10) , d is  the  relative  distance 
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2.3  Qualitative  view  of  {d(IQM)/dt}/{IQM} 

As  shown  in  the  previous  sections  the  VTC  is  defined  only  in  a region  beyond  a certain 
desired  minimum  clearance  Rq  and  is  not  defined  when  the  distance  between  the  camera  is 
less  than  Rq.  Though  we  restrict  ourselves  to  regions  beyond  the  desired  minimum 
clearance  there  might  be  situations  when  one  is  in  the  region  for  which  the  distance 
between  the  camera  and  the  surface  is  less  than  Rq.  Since  the  VTC  is  undefined  in  this 
region  the  VTC  cannot  be  employed  when  the  robot  is  in  this  region.  However  the  IQM 
and  relative  temporal  variations  in  IQM  ({d(IQM)/dt}/{IQM})  can  be  used  since  it  is  an 
image  measure  and  is  defined  irrespective  of  the  distance  between  the  camera  and  the 
surface.  In  this  section  we  present  an  overall  qualitative  behavior  of  the  IQM  as  well  as 
{d(IQM)/dt}/{IQM}. 

Consider  the  case  of  a fixed  focus  camera  that  is  initially  focused  to  a 3D  surface  at  a 
very  short  distance  say  Rq.  As  the  camera  moves  away  from  the  surface  (R  > Rq)  or 
moves  towards  the  surface  (R<Rq)  the  perceived  2D  image  texture  varies  from  one  image 
to  another  mainly  due  to  focus.  The  details  in  the  image  get  smeared  when  R is  not  equal 
to  Rq.  As  the  details  get  smeared  the  dissimilarity  of  the  image  becomes  smaller  and 
smaller.  A plot  depicting  the  qualitative  behavior  of  the  IQM  versus  the  distance  between 
the  camera  and  the  surface  is  shown  in  Figure  (5).  Experimental  results  to  support  this 
qualitative  behavior  of  the  IQM  which  is  basically  a focus  measure  can  be  found  in  [44, 
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45].  A qualitative  plot  of  {(d(IQM)/dt)}/{IQM}  for  a given  speed  is  also  shown  in  Figure 

(5). 


Figure  5:  Qualitative  IQM  and  {(d(IQM)/dt)}/{IQM}for  a given  speed 
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3 Control  Schemes 

In  this  section  we  describe  the  vision-based  fu2jzy  control  schemes  employed  to 
achieve  the  tasks  of  collision  avoidance  and  maintenance  of  clearance  using  the 
{(d(IQM)/dt)}/{IQM}  as  sensory  feedback  signal. 

In  this  section  we  present  two  vision-based  fiizzy  control  schemes  to  accomplish  the 
following  tasks: 

Task  1:  Collision  Avoidance:  The  task  is  to  stop  a robot  in  motion  in  front  of  a 
textured  surface  when  the  relative  distance  between  the  robot  and  the  surface  is  a desired 
one  using  visual  information  only. 

Task  n:  Maintenance  of  Clearance:  The  task  is  to  follow  a textured  surface  using 
visual  information  only. 


Figure  6:  Block  diagram  of  the  control  scheme 


3.1  Control  Scheme  I:  This  control  scheme  has  been  employed  to  achieve  task  I.  Initially 
the  camera  is  focused  to  distance  which  is  equal  to  Rq.  For  ranges  R greater  than  Rq,  as 
the  range  increases  the  IQM  (denoted  as  C from  now  on)  value  decreases  and  vice-versa. 
Based  on  the  relative  temporal  variations  in  the  IQM  value,  we  divide  the  region  in  front 
of  the  mobile  robot  into  three  different  regions  as  shown  in  Figure  (2).  Region  I can  be 
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seen  as  a safe  region  (refer  to  Figure(7))  and  regions  11  and  in  can  be  seen  as  danger 
zones. 

The  control  actions  are  based  on  the  fuzzy  relative  temporal  changes  in  IQM  and  can 
be  described  by  the  following  rules,  (d(.)/dt  indicates  differentiation  of  (.)  with  respect  to 
time,  {d(IQM)/dt}/{IQM}  = {d(C)/dt}/{C}),  (assume  t2  >tl): 

Rule  I:  This  rule  corresponds  to  the  case  when  the  robot  is  in  the  safe  zone  (region  I 
in  Figure  (7)).  In  this  zone,  no  control  action  should  be  taken,  i.e.,  no  change  in  speed  is 
necessary.  The  sensing  and  action  corresponding  to  this  region  can  be  expressed  in  the  IF- 
THEN  format  as  follows. 


If 


( • A 
C 

> 0 and 

( • A 

C 

> 0 and 

( • A 
C 

> 

( • A 
C 

C 

V y 

t=t2 

C 

A y 

t=t\ 

C 

A y 

t=t2 

C 

A y 

Then  No  Action 


Note: 


( • A 

( • A 

c 

>0  and 

c 

c 

C 

A y 

t-t2 

A y 

> 0 indicates  the  motion  of  the  robot  towards  the 


t-a 


surface  when  the  robot  is  in  the  region  beyond  the  desired  minimum  clearance. 


Rule  Hi  This  rule  corresponds  to  region  n in  Figure  (7).  In  this  region  it  is  required  to 
stop  the  moving  robot  if  the  robot  is  moving  towards  the  surface  and  if  it  crosses  the 
desired  clearance.  The  motion  of  the  robot  towards  the  surface  for  this  region  can  be 
expressed  in  an  IF-THEN  format  as  follows. 


IF 


( • A 
C 

C 

V y 


j=»i 


> Threshold  and 


( • A 
C 

C 

V y 


f=r2 


<0 


Then  Stop 
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Rule  ni:  This  rule  corresponds  to  region  IQ  in  Figure  (7).  In  this  region  the  robot  is 

( • ^ 

C 

required  to  stop  if  it  is  moving  towards  the  surface.  Note  that  in  this  region  — <0.  The 

V 7 

motion  of  the  robot  towards  the  surface  for  this  region  can  be  expressed  in  the  IF-THEN 
format  as  follows: 


If 


( • ^ 

£ 

C 

V 


< 0 and 

( • ^ 
c 

< 0 and 

( • A 
£ 

> 

( * ^ 
£ 

»2 

c 

/=U 

C 

/=fi 

C 

1 y 

Then  Stop. 


»=r2 


Rule  rV:  1¥  none  of  the  above  situations  occur  THEN  take  no  action. 

Usually  this  situation  arises  when  the  robot  is  either  stationary  or  moving  away  from 
the  surface. 

AH  four  rules  for  the  control  scheme  I can  be  summarized  as  follows: 


Rule 

Sensing 

Action 

1 

IF 

^ ^ 
C 

C 

V y 
f • ^ 

£ 

C 

V > 

> 0 am 

t=t2 

f . > 

c 

> — 
c 

t=t2  V Z 

1 

^ 

C 

c 

^ y 

(t2 

j 

> 0 and 

t=ii 

>tl) 

THEN 

No  Action 

2 

IF 

^ ^ 

C 

c 

< y 
f • ^ 
£ 

C 

V > 

> Threshold  and 

r=»l 

<0  (t2>tl) 

t=t2 

THEN 

Stop 

3 

IF 

1 

^ ^ 

c 

c 

^ y 

uid 

< 

t=/2 
^ ^ 
£ 
C 

^ y 

c 0 am 

> 

[] 

£ 

C 

r 

K 

y 

. \ 

£ 

C 

y 

»=f2 

<0 

/=rl 

(t2  > tl) 

THEN 

Stop 

4 

IF 

ELSE 

THEN 

No  Action 

Table  1:  Summary  of  rules  for  vision-based  collision  avoidance 
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Figure  7:  Qualitative  plot  of  the  relative  variations  of  IQM  {d(C)/dt}/{C} 


3.2  Control  Scheme  11:  This  control  scheme  has  been  employed  to  accomplish  task  H, 
i.e.,  maintenance  of  clearance.  Refer  to  Figures  (8a)  and  (8b). 
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Figure  8a:  Region  A corresponds  to  the  left  of 
the  camera  and  region  B corresponds  to  the  right  of 
the  camera 


Figure  8b:  The  image 
corresponding  to  Figure  (8a). 


In  Figure  (8a)  the  left  region  (region  A)  is  closer  to  the  camera  than  the  right  region 
(region  B).  The  camera  is  initially  focused  at  a desired  minimum  clearance  Rq.  As 
mentioned  earlier,  when  the  distance  between  the  camera  and  the  surface  is  greater  than 
the  desired  minimum  clearance,  the  points  located  at  a greater  distance  have  a relatively 
smaller  relative  temporal  variations  in  the  IQM,  i.e.,  {d(C)/dt}/{C}  than  those  located  at  a 
relatively  smaller  distance.  The  computer  compares  the  {d(C)/dt}/{C}  of  the  left  window 
as  shown  in  Figure  (8b)  with  the  {d(C)/dt}/{C}  of  the  right  window  This  is  the  key  to 


generate  steering  commands  to  the  robot,  i.e.,  the  difference  between  ( — ) corresponding 

.Left 


to  the  left  window  (denoted  as  — 


) and  ( — ) corresponding  to  the  right  window 


(denoted  as  — 


Right 


) is  the  control  signal  for  control  scheme  n.  Note  that  a change  in  the 


direction  of  motion  of  the  robot  can  be  accomplished  by  changing  forward,  lateral  or  both 
components  of  velocity  (as  shown  in  Figure  (8a)).  When  the  robot  is  in  a region  beyond 
the  desired  minimum  clearance  the  change  in  heading  direction  is  generated  by  controlling 
only  the  lateral  component  When  the  robot  is  in  the  clearance  zone  both  lateral  as  well  as 
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forward  velocity  components  are  controlled.  The  control  scheme  can  be  explained  by  IF- 
THEN  rules  as  follows  (assume  t2  > tl): 


• 

Left  , 

Left 

Right  , 

Right  , 

Left  , 

c 

c 

, c 

c 

, X 

c 

> — 

and  — 

> — 

and  ( — 

c 

c 

C 

c 

C 

c 

t=t2 

t=ti 

t=t2 

t=ti 

t=t2 

Right 


) is  approximately 


t=t2 


zero  THEN  no  change  in  the  velocity. 

According  to  this  rule  when  the  robot  approaches  the  surface  and  if  both  the  left  and 
right  regions  are  very  far  from  the  robot  then  there  is  no  change  in  the  current  velocity. 
Region  A and  region  B are  beyond  the  desired  minimum  clearance. 


• 

Left  , 

Left 

Right  , 

Right  , 

Left  . 

c 

c 

, c 

c 

. X 

c 

> — 

and  — 

> — 

and  ( — 

c 

c 

C 

c 

C 

c 

t=t2 

t=ti 

t=t2 

t=ti 

t=t2 

Right 


) is  small  THEN 


t=t2 


motion  to  the  right  is  small. 


• 

Left 

Left 

Right  , 

Right  , 

Left  ^ 

c 

c 

, c 

c 

. X 

c 

> — 

and  — 

> — 

and  ( — 

c 

c 

C 

c 

C 

c 

t=t2 

t=ti 

t=t2 

t=ti 

t=t2 

Right 


) is  medium 


t=t2 


THEN  motion  to  the  right  is  medium. 


• 

Left 

Left 

Right  , 

Right  , 

Left  , 

c 

c 

, c 

c 

. X 

_c 

> — 

and  — 

> — 

and  {— 

c 

c 

C 

c 

C 

c 

t=t2 

t=ti 

t=t2 

t=ti 

t=t2 

Right 


) is  big  THEN 


t=t2 


motion  to  the  right  is  big. 


• 

Left  , 

Left 

Right  , 

c 

c 

^ c 

c 

< — 

and  — 

> — 

c 

c 

t=t2 

C 

t=ti 

c 

t=t2 

Right 


t=tl 


THEN  motion  to  the  right  is  big  and 


reverse  the  current  direction  of  motion. 
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According  to  this  rule  when  the  region  A is  within  the  desired  clearance  and  region  B 
is  in  the  region  beyond  the  desired  clearance,  the  desired  control  is  to  move  to  the  right 
and  move  backwards. 


* 

Left  , 

Left 

Right  , 

c 

c 

, c 

c 

< — 

and  — 

< — 

c 

c 

C 

c 

t=t2 

t=ti 

t=t2 

Right 


t=tl 


THEN  motion  to  the  right  is  big  and 


reverse  the  current  direction  of  motion. 

According  to  this  region  both  region  A and  region  B are  within  the  desired  clearance 
region.  The  desired  control  action  is  to  move  the  robot  backwards. 


Rule  Vn:  IF  none  of  the  above  situations  occur  THEN  take  no  change  in  the 
velocity. 

Usually  this  situation  arises  when  the  robot  is  either  stationary  or  moving  away  from 
the  surface. 


3.3  Membership  Functions 

In  this  section  we  present  a qualitative  view  of  the  membership  functions  employed  for 
fuzzification. 

3.3a  Control  Scheme  I:  Since  only  two  modes  of  operation  are  necessary  in  this  control 
scheme,  no  membership  functions  are  employed  in  this  control  scheme. 

3.3b  Control  Scheme  11;  In  this  the  section  the  membership  functions  used  in  the  Control 
Scheme  n are  described.  We  employ  linear  membership  functions  as  shown  in  Figure  (9). 


18 


Figure  9:  Qualitative  Membership 
functions  used 


Figure  10:  Evaluation  of  weight  of  a 
particular  rule 


3.4  Defuzzification 

Defuzzification  of  the  inferred  fuzzy  control  action  is  necessary  in  order  to  produce  a 
crisp  control  action.  Since  monotonic  membership  functions  are  used,  we  use  Tsukamoto's 
defuzzification  method,  which  is  stated  as  follows: 


i=l 


where  Z*  is  the  defuzzified  crisp  control  command  and  aj  is  the  weight  corresponding 
to  the  rule  i;  y[  is  the  amount  of  control  action  recommended  by  rule  i and  n is  the  number 
of  rules. 

We  used  the  ratio  of  the  shaded  area  in  Figure  (10)  to  the  area  of  the  triangle  as  the 
firing  strength  and  is  derived  to  be: 


^i  ~ Pi(2‘Pi) 


when  Pi  equals  1,  the  shaded  area  equals  the  area  of  the  triangle,  hence  ai  is  1 
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4 Implementation  Details 

The  control  algorithms  presented  in  the  previous  sections  are  implemented  on  a 6- 
DOF  vision-based  flight  simulator  controlled  by  a 486  based  Personal  Computer.  A CCD 
video  camera  is  used  to  obtain  the  images  of  the  3D  textured  environments.  These  images 
are  digitized  by  an  image  processing  PC-board  TTEX  PC- VISION  PLUS.  A block 
diagram  of  the  experimental  setup  is  shown  in  Figure  (11a).  Photo  copies  of  texture 
pattern  (D5,  DUO  from  [42],  see  Figure  (12))  pasted  on  a flat  surface  is  presented  as  the 
obstacle  along  the  path  of  the  robot  (see  Figure  (lib)).  For  both  control  schemes  the 
camera  is  initially  focused  to  the  desired  minimum  clearance  (Rq  = 200  mm.).  We  use 
qualitative  measures  for  fuzzy  sensing  and  action  (small,  medium,  big,  etc.)  rather  than  the 
exact  speeds. 


SCENE 

VIEW 


VIDEO 

MONITOR 


<J= 


SIMULATOR 


PC 

MONITOR 


IMAGE  PROCESSOR 

486/50MHZ  BASED 
PERSONAL 
COMPUTER 


ANALOG 

OUTPUT 

SIGNALS 


ROBOT 

CONTROLLER 


6-DOF  POSITION* 
ORIENTATION 


Figure  11a:  Block  diagram  of  implementation 
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Figure  lib:  Camera  mounted  on  the  Robot 


Figure  12:  Textures  used  in  the  experiments 


4.1  Control  Scheme  I 

A window  of  size  50  X 50  pixels  is  chosen  in  the  center  of  the  image  to  evaluate  the 
visual  feedback  signal  {d(C)/dt}/{C}.  According  to  the  rules  presented  in  the  previous 
section  the  crisp  control  action  is  (either  move  or  stop)  is  generated.  Two  different  speeds 
were  employed  in  this  control  scheme  (speed2  > speed  1). 

4.2  Control  Scheme  n 

Two  windows  (left  and  right)  each  50  X 50  pixels  are  opened  in  the  image.  In  each  of 
these  windows,  the  visual  parameter  {d(C)/dt}/{C}  is  evaluated  and  based  on  the 
difference  between  left  and  right  values  an  appropriate  control  signal  is  generated.  This 
control  scheme  was  tested  for  four  different  orientations  of  the  texture  surfaces  used. 
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5 Results  and  Analysis 

In  this  section  we  present  the  results  and  analysis  of  the  control  schemes  are 
presented. 

5.1  Control  Scheme  I 

Two  different  speeds  were  used  to  test  the  braking  capability  of  the  control  algorithm. 
We  observed  that  the  greater  the  speed  of  the  robot,  the  greater  is  the  error  between  the 
desired  and  actual  values  of  the  clearance  between  the  robot  and  the  surface  (see  Figure 
(13)).  The  results  can  be  summarized  as  follows: 


No. 

Texture 

Speed 

Desired. 

Actual 

Error 

1 

D5 

Speed  1 

200  mm, 

180  mm. 

20  mm. 

2 

D5 

Speed2 

200  mm. 

165  mm. 

35  mm. 

3 

DUO 

Speed 1 

200  mm. 

180  mm. 

20  mm 

4 

DUO 

Speed2 

200  mm. 

165  mm. 

35  mm. 

Table  2:  Summary  of  vision-based  collision  avoidance  results 


5.2  Control  Scheme  II 

The  lateral  and  longitudinal  components  of  the  heading  vector  were  recorded.  The 
resultant  was  plotted  manually  (see  Figures  (14-21)).  AU  the  four  experiments  in  this 
control  scheme  employed  the  same  rule  base.  The  error  between  the  desired  path  and  the 
actual  path  is  highly  dependent  upon  the  choice  of  fuzzy  membership  functions,  rule-base 
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and  defuzzification  schemes  used.  Addition  of  more  rules  to  the  existing  ones  may 
improve  the  error  between  the  desired  and  actual  paths. 


Figure  14:  Maintenance  of  Clearance:  Actual  Data  for  Texture  D1 10 
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Figure  15:  Maintenance  of  Clearance:  Actual  Data  for  Texture  DUO 
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Figure  16:  Maintenance  of  Clearance:  Actual  Data  for  Texture  DUO 


Figure  17:  Maintenance  of  Clearance:  Actual  Data  for  Texture  DUO 
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Figure  18:  Maintenance  of  Clearance:  Actual  Data  for  Texture  D5 
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Figure  19:  Maintenance  of  Clearance:  Actual  Data  for  Texture  D5 
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Figure  20:  Maintenance  of  Clearance:  Actual  Data  for  Texture  D5 
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Figure  21:  Maintenance  of  Clearance:  Actual  Data  for  Texture  D5 
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6 Conclusions  and  Future  Work 

This  paper  presented  two  vision-based  fuzzy  logic  control  schemes  using  relative 
temporal  variations  in  the  image  dissimilarity  for  the  tasks  of  collision  avoidance  and 
maintenance  of  clearance.  The  use  of  relative  temporal  variations  of  image  dissimilarity 
needs  no  optical  flow  information,  segmentation  or  feature  tracking.  These  control  scheme 
demonstrate  the  use  of  vision  as  a sensory  feedback  signal  in  unknown  environments  for 
two  navigation  tasks.  The  control  schemes  are  approximate  and  do  not  take  into  account 
the  robot  dynamics.  Currently  we  are  developing  a scheme  that  takes  into  account  the 
temporal  sampling  and  robot  dynamics.  This  paper  deals  with  linear  membership  functions 
only.  Several  other  monotonic  functions  are  currently  being  employed.  It  is  also  possible 
to  implement  classical  control  techniques  like  PD,  PID  control  and  use  a fuzzy  logic 
controller  as  a supervisor  to  monitor  the  PD,  PID  control  schemes. 
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